2 links tagged with all of: mixture of experts + reasoning
Click any tag below to further narrow down your results
Links
DeepSeek V4.1 Flash is a 552B-parameter mixture-of-experts model that activates only 8B-16B parameters per token and supports a 1M-token context window for handling long documents and agentic workloads. The model cuts KV-cache memory usage by 4x compared to its predecessor while improving coding, reasoning, and agent performance.
- Activates just 8B parameters during input processing and 16B during generation, making it efficient despite the massive 552B backbone
- Reduces memory footprint by roughly 4x versus DeepSeek-V4-Flash-0731 through Compressed Sparse Attention 2 and a Causal Encoder-Decoder architecture
- Offers configurable reasoning-effort levels so developers can adjust latency and cost versus reasoning depth
- Handles multimodal input (text and images) natively with a 1M-token context window, designed for agents processing large amounts of data
The article introduces Inkling-Small, a Mixture-of-Experts transformer with 276 B total and 12 B active parameters that matches full Inkling performance at one-quarter the size. It handles text, images, and audio with a 1 M-token context window and lets users trade compute for thinking effort. The open-weight model outperforms peers on tool use, coding, reasoning, and multimodal benchmarks and is available for fine-tuning and chat.
- Inkling-Small matches or beats full Inkling performance (e.g., 31.6% vs 29.7% on Humanity's Last Exam) while using only 12B active params out of 276B total—roughly a quarter the compute cost.
- It's natively multimodal (text, images via 40×40 patches, audio via dMel spectrograms) with a 1M-token context window and adjustable "thinking effort" for cost/performance tradeoffs.
- Safety numbers slightly lag full Inkling (71.6% vs 78.0% harmful-query refusal on FORTRESS) despite reusing the same safety recipe.
- Open-weight and available now for fine-tuning on Tinker and chat testing in the Tinker Playground.